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We claim: 



1, A method of encoding a sequence of images for transmission over a 
communication network, said method comprising the steps of: 



rendering an first image from a object-based computer graphics 
encoding said first image according to a predetermined encoding 
rendering an second image from said object-based computer graphics 



(i) 

application; 

(ii) 
scheme; 4 

(iii) 
application; 

(iv) identifying changes between said first image and said second image 
from a change indicating output of said computer graphics application; 

(v) using said identified changes to determine a manner in which said 
second image is encoded according to said predetermined format; and 

(vi) repeating steps (iii) to (v) for each subsequent image of said sequence. 



2. A method according to claim 1 wherein said computer graphics application 
outputs a pixel-based representation of each said image to a buffer, and said encoding, at 
least for said first image, operates upon said pixel-based representation and for said 
second and subsequent images upon at least said change indicating output. 



3. A method according to claim 2 wherein said change indicating output comprises 

information indicating an extent of change in said pixel-based representation in said 
buffer thereby enabling said encoding according to said predetermined encoding scheme 
of substantially only those pixels that have changed. 



4. A method according to claim 2 wherein said computer graphics application acts 
to render a hierarchical representation of each said image and said change indicating 
output comprises information indicating an extent of change between said images of the 
corresponding said hierarchical representations. 



5. A method according to claim 1 wherein said predetermined encoding scheme 

comprises MPEG encoding and said encoding fonns, for each said image, a data stream 
for transmission over said network. 
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6. An interactive system in which user commands at a first location are 
communicated via a communications network to a processing arrangement at a second 
location, said processing arrangement interpreting said commands and in response thereto 
transmitting a sequence of images for display at said first location to said user, each said 
image being encoded to form at least a component of a data stream, characterised in that 
each said image is formed at said processing arrangement from a graphic object 
representation of said image and, in response to a said user command, a rendering of said 
graphic object representation is used to encode said image and form said stream. 

7. A system according to claim 6 wherein said rendering is used exclusively as an 
encoding source and from which said stream is formed. 

8. An encoder for encoding a series of images into a bitstream, each said image 
being rendered from a graphics object application characterised in that said encoder is 
constrained to operate according to a plurality of inputs describing the image, where the 
format of said inputs is known by the encoder. 

9. An encoder according to claim 8 wherein one of said inputs comprises a first 
change input representing those portions of a pixel map of a current said image that have 
changed relative to an immediately preceding image in said series. 

10. An encoder according to claim 8 wherein an input to said graphical object 
application comprises a hierarchical representation of said image and one of said inputs 
comprises a second change input representing those portions of said hierarchical 
representation of a current said that have changed relative to an immediately preceding 
image in said series. 

11. A system for manipulating user commands over a computer network, said 
system comprising: 

a user terminal coupled to said network at a first location and incorporating 
means for receiving commands from said user and display means for displaying images; 

a server computer coupled to said network at a second location, said server 
computer comprising a computing application arranged to receive and process said 
commands from said user terminal, and an output generator for forming an output display 
of said computing application; 
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characterised in that 

said output generator comprises a graphic object application for rendering 
images formed by said computing application and an encoder for forming an encoded 
bitstream output of said rendered images for communication over said network for 
display on said display means of said user terminal, and said user terminal comprises a 
decoder for decoding said bitstream for display on said display means. 

12. A system according to claim 1 1 characterised in that said computing application 
comprises a browsing application for browsing said computer network. 

13. A system according to claim 1 1 characterised in that said bitstream comprises an 
MPEG stream. 

14. A system according to claim 13 characterised in that said bitstream consists of an 
MPEG stream. 

15. The invention according to any one of claims 5, 7 5 10 or 14 wherein said 
encoding comprises MPEG encoding and incorporates at least one of the optimisations 
substantially as described herein. 

16. A method of forming a sequence of images for transmission over a 
communication network, said method comprising the steps of: 

forming representations of plural graphical object-based images; 
rendering said images in a sequence using a computer graphics application, said 
rendering comprising: 

determining a set of changes required to transform a previous image in 
said sequence into a current image in said sequence; 

indicating said changes in a format known to an encoder a priori; 
encoding said sequence usmg said encoder such that said encoder utilises at least 
one of said changes to optimise encoding of said current image. 

17. A method according to claim 16 wherein said representations comprise at least 
one hierarchical compositing tree. 
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18. A method according to claim 16 wherein said at least one of said changes is 
other than a pixel map representation of said current image. 

19. A method according to claim 18 wherein one said change comprises information 
related to a transformation matrix of changed regions in said current image. 

20. A method according to claim 18 wherein one said change comprises information 
regarding content of said current image. 

21. A method according to claim 20 wherein said content comprises at least one of 
plane fill data and a run-length encoded data used to form said current image and said 
information indicates that said content forms a region of flat colour in said current image. 

22. A method according to claim 21 wherein said content further comprises a pixel 
map of said current image. 

23. A method according to claim 18 wherein one said change comprises information 
regarding whether a region of said current image is one of a background region or a 
foreground region. 

24. A method according to claim 18 wherein one said change comprises information 
regarding a position and area of a region of said current image that has changed. 

25. A method according to claim 18 wherein one said change comprises information 
regarding changes to said compositing tree. 

26. A method according to claim 21 wherein, when said content comprises a plane 
fill, said encoding comprises a single colour conversion operation and encoding of a 
single pixel value for said current image. 

27. A method according to claim 21 wherein, when said content comprises run- 
length data, said encoding comprises, for each run of data in said current image, a single 
conversion and encoding of a pixel value. 
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28. A method according to claim 16 further comprising storing an encoded 
representation of at least said current image for use in an encoding of a subsequent image 
in said sequence. 

29. A method according to claim 28 wherein said encoded representation comprises 
a plurality of separately encoded discrete portions. 

30. A method according to claim 29 wherein said set of changes comprises a change 
to said compositing tree, said change indicating a portion of a rendered current image 
having changed due to motion in the sequence, said encoding comprising determining 
those of said discrete portions that require update in view of said change, encoding 
rendered pixel values output from said computer graphics application corresponding to 
said changes discrete portions as replacement ones of said portions, and combining said 
replacement portions with remaining portions of the preceding encoded image to form a 
current encoded image. 

31. A method according to claim 29, further comprising, for a first image of a 
determinable sequence, storing an encoded version of said first image with a flag 
identifiable by said indicated change such that said indicate change causes retrieval of 
said stored encoded first image for transmission. 

32. A method according to claim 31 wherein said indicated change for a subsequent 
rendered image in said sequence is compared with said store encoded first image and 
wherein said encoded first image is stored as a plurality of separately encoded portions 
and said indicated change for said subsequent rendered image is use to encode only those 
said portions that have changed. 

33. A method according to claim 28 wherein if said current image is identical to said 
preceding image, said encoding comprises encoding a special image indicator 
representative of no-change in said sequence at said current image. 

34. A method according to claim 33 wherein said encoded preceding image 
comprises a plurality of slices in raster scan order each formed of plural of said discrete 
portions, said indicator being provided for a slice of said image, wherein those said 
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discrete portions that have not changed are encoded within each said slice using motion 
vectors set to zero. 

35, A method according to claim 29 wherein said encoding forms an MPEG 
representation of each said image in which said discrete portions comprise macroblocks. 

36, A method according to claim 35 wherein said encoding forms an MPEG P- 
frame, said encoding further comprising: 

(a) extracting a slice of a rendered image frame; 

(b) determining if a first macroblock of said extracted slice is dirty and, if 
so, encoding said first macroblock to an output bitstream and storing a copy of said 
encoded first macroblock in a cache and, if not, copying a pre-encoded macroblock from 
said cache to said output bitstream; 

(c) for each further macroblock in said slice barring the last, determining if 
said macroblock is dirty and, if so: 

(ca) encoding the macroblock to said output bitstream; 

(cb) storing the encoded macroblock in said cache; 

(d) for the last macroblock in said slice 

(da) determining if said macroblock is dirty; 

(db) if so, encoding the last macroblock to said output bitstream; 

(dc) if not, copying a pre-encoded macroblock from said cache to 
said output bitstream; and 

(e) repeating steps (a) to (d) for each remaining slice of said rendered image 
frame to thereby form said P- frame, 

37, A method according to claim 35 wherein said encoding forms an MPEG I-frame, 
said encoding further comprising: 

(a) obtaining a macroblock of a rendered image frame; 

(b) extracting an unencoded DC value of the previous macroblock from a 
cache used to store encoded frames; 

(c) determining a difference DC value between the obtained macroblock 
and the extracted unencoded DC value; 

(d) encoding the difference DC value to an output bitstream; 

(e) copying an encoded AC component from said cache to said output 
bitstream; and 
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(f) repeating steps (a) to (c) for each macroblock of said rendered image 
frame to form said I-frame. 



38. A method according to claim 35 wherein said encoding further comprises setting 
5 a flag on any macroblock that has previously been marked as dirty and subsequently 
encoded and upon any previous said macroblock that has said flag set, said flag being 
used to determine whether one of cached bitstream representation of said macroblock or a 
stored encoded bitstream of said macroblock is restored to an output bitstream. 



10 39. A method according to claim 35 wherein said encoding comprises, for each said 
Mb macroblock: 

(a) storing a DC component for said macroblock; 
ft! (b) storing a reference from said macroblock to an immediately preceding 

macroblock; 

m 

15 (c) DCT transforming, quantising and storing an AC component of said 

macroblock; and 

y : (d) storing a flag to mark said macroblock as either dirty or non-dirty. 

m 

i .. 

gl 40. A method according to claim 39, further comprising, for each macroblock that is 

O 20 marked as dirty, the steps of: 

(i) applying a DCT; 

(ii) calculating and saving the DC delta ; 

(iii) quantising the macroblock; 

(iv) saving the DC component of said macroblock; 

25 (v) encoding the macroblock using bit stream encoding; 

(vi) saving the bitstream encoded macroblock; and 

(vii) setting dirty flag to not dirty. 



4L A method according to claim 39, further comprising, for each macroblock that is 
30 marked as non-dirty, and is not preceded by dirty blocks since the start of a current slice, 
restoring the macroblock from a bitstream encoded version thereof. 

42. A method according to claim 39, further comprising, for each macroblock that is 
not marked as dirty, but preceded by a macroblock marked as dirty since the start of a 
35 current slice, the steps of: 
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(i) restoring the cached DC values; 

(ii) calculating the DC delta; 

(Hi) quantising the DC portion of the macroblock; 

(iv) encoding the macroblock using bit stream encoding; and 

(v) saving the bitstream encoded macroblock. 

43. A computer readable medium having recorded thereon a computer program for 
forming a sequence of images for transmission over a communication network, said 
program comprising: 

code for forming representations of plural graphical object-based images; 
code for rendering said images in a sequence using a computer graphics 
application, said code for rendering including: 

code for determining a set of changes required to transform a previous 
image in said sequence into a current image in said sequence; 

code for indicating said changes in a format known to an encoder a 

priori; 

code for encoding said sequence using said encoder such that said encoder 
utilises at least one of said changes to optimise encoding of said current image. 

44. A computer readable medium according to claim 43 wherein said representations 
comprise at least one hierarchical compositing tree. 

45. A computer readable medium according to claim 43 wherein said at least one of 
said changes is other than a pixel map representation of said current image. 

46. A computer readable medium according to claim 45 wherein one said change 
comprises information related to a transformation matrix of changed regions in said 
current image. 

47. A computer readable medium according to claim 45 wherein one said change 
comprises information regarding content of said current image. 

48. A computer readable medium according to claim 47 wherein said content 
comprises at least one of plane fill data and a run-length encoded data used to form said 
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current image and said information indicates that said content forms a region of flat 
colour in said current image. 

49. A computer readable medium according to claim 48 wherein said content further 
5 comprises a pixel map of said current image. 

50. A computer readable medium according to claim 45 wherein one said change 
comprises information regarding whether a region of said current image is one of a 
background region or a foreground region. 



10 



51. A computer readable medium according to claim 45 wherein one said change 
comprises information regarding a position and area of a region of said current image that 

iw has changed. 

Q 
yy 

tfl 15 52. A computer readable medium according to claim 45 wherein one said change 
9 comprises information regarding changes to said compositing tree. 

fl| 

53. A computer readable according to claim 48 wherein, when said content 
W comprises a plane fill, said encoding comprises a single colour conversion operation and 

20 encoding of a single pixel value for said current image. 

54. A computer readable medium according to claim 48 wherein, when said content 
comprises run-length data, said encoding comprises, for each run of data in said current 
image, a single conversion and encoding of a pixel value. 

25 

55. A computer readable medium according to claim 43 further comprising code for 
storing an encoded representation of at least said current image for use in an encoding of 
a subsequent image in said sequence. 

30 56. A computer readable medium according to claim 55 wherein said encoded 
representation comprises a plurality of separately encoded discrete portions. 

57. A computer readable medium according to claim 56 wherein said set of changes 
comprises a change to said compositing tree, said change indicating a portion of a 
35 rendered current image having changed due to motion in the sequence, said code for 
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encoding comprising code for determining those of said discrete portions that require 
update in view of said change, code for encoding rendered pixel values output from said 
computer graphics application corresponding to said changes discrete portions as 
replacement ones of said portions, and code for combining said replacement portions with 
remaining portions of the preceding encoded image to fonn a current encoded image. 

58. A computer readable medium according to claim 56, further comprising, for a 
first image of a determinable sequence, code for storing an encoded version of said first 
image with a flag identifiable by said indicated change such that said indicate change 
causes retrieval of said stored encoded first image for transmission. 

59. A computer readable medium according to claim 58 wherein said indicated 
change for a subsequent rendered image in said sequence is compared with said store 
encoded first image and wherein said encoded first image is stored as a plurality of 
separately encoded portions and said indicated change for said subsequent rendered image 
is use to encode only those said portions that have changed. 

60. A computer readable medium according to claim 55 wherein if said current 
image is identical to said preceding image, said code for encoding is operative to encode a 
special image indicator representative of no-change in said sequence at said current 
image. 

61. A computer readable medium according to claim 60 wherein said encoded 
preceding image comprises a plurality of slices in raster scan order each formed of plural 
of said discrete portions, said indicator being provided for a slice of said image, wherein 
those said discrete portions that have not changed are encoded within each said slice 
using motion vectors set to zero. 

62. A computer readable medium according to claim 56 wherein said encoding 
forms an MPEG representation of each said image in which said discrete portions 
comprise macroblocks. 

63. A computer readable medium according to claim 62 wherein said encoding 
forms an MPEG P-firame, said encoding further comprising: 

(a) extracting a slice of a rendered image frame; 
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(b) determining if a first macroblock of said extracted slice is dirty and, if 
so, encoding said first macroblock to an output bitstream and storing a copy of said 
encoded first macroblock in a cache and, if not, copying a pre-cncoded macroblock from 
said cache to said output bitstream; 

(c) for each further macroblock in said slice barring the last, determining if 
said macroblock is dirty and, if so: 

(ca) encoding the macroblock to said output bitstream; 

(cb) storing the encoded macroblock in said cache; 

(d) for the last macroblock in said slice 

(da) determining if said macroblock is dirty; 

(db) if so, encoding the last macroblock to said output bitstream; 

(dc) if not, copying a pre-encoded macroblock from said cache to 
said output bitstream; and 

(e) repeating steps (a) to (d) for each remaining slice of said rendered image 
frame to thereby form said P-frame. 

64. A computer readable medium according to claim 61 wherein said encoding 
fomxs an MPEG I-frame, said encoding further comprising: 

(a) obtaining a macroblock of a rendered image frame; 

(b) extracting an unencoded DC value of the previous macroblock from a 
cache used to store encoded frames; 

(c) determining a difference DC value between the obtained macroblock 
and the extracted unencoded DC value; 

(d) encoding the difference DC value to an output bitstream; 

(e) copying an encoded AC component from said cache to said output 
bitstream; and 

(f) repeating steps (a) to (e) for each macroblock of said rendered image 
frame to form said I-frame, 

65. A computer readable medium according to claim 61 wherein said encoding 
further comprises setting a flag on any macroblock that has previously been marked as 
dirty and subsequently encoded and upon any previous said macroblock that has said flag 
set, said flag being used to determine whether one of cached bitstream representation of 
said macroblock or a stored encoded bitstream of said macroblock is restored to an output 
bitstream. 
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66, A computer readable medium according to claim 61 wherein said encoding 
comprises, for each said macroblock: 

(a) storing a DC component for said macroblock; 

(b) storing a reference from said macroblock to an immediately preceding 
macroblock; 

(c) DCT transforming, quantising and storing an AC component of said 
macroblock; and 

(d) storing a flag to mark said macroblock as either dirty or non-dirty, 

67. A computer readable medium according to claim 66, further comprising, for each 
macroblock that is marked as dirty, the steps of: 

(i) applying a DCT; 

(ii) calculating and saving the DC delta ; 
(in) quantising the macroblock; 

(iv) saving the DC component of said macroblock; 

(v) encoding the macroblock using bit stream encoding; 

(vi) saving the bitstream encoded macroblock; and 

(vii) setting dirty flag to not dirty. 

68, A computer readable medium according to claim 66, further comprising, for each 
macroblock that is marked as non-dirty, and is not preceded by dirty blocks since the start 
of a current slice, code for restoring the macroblock from a bitstream encoded version 
thereof. 

69. A computer readable according to claim 65, further comprising, for each 
macroblock that is not marked as dirty, but preceded by a macroblock marked as dirty 
since the start of a current slice, the steps of: 

(i) restoring the cached DC values; 

(ii) calculating the DC delta; 

(iii) quantising the DC portion of the macroblock; 

(iv) encoding the macroblock using bit stream encoding; and 

(v) saving the bitstream encoded macroblock. 
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70. Apparatus for fonning an encoded a sequence of images for transmission over a 
communication network, said apparatus comprising: 

a graphics processor for forming representations of plural graphic object-based 

images; 

a renderer for rendering each said representation in sequence to form each said 
image, said renderer determining for a current image in said sequence excepting the first, 
a set of changes necessary to transform an immediately preceding image in said sequence 
into said current image; and 

an encoder for receiving each said rendered current image and corresponding set 
of changes and for encoding said image as part of said sequence into a bitstream for said 
transmission, said encoder using at least one member of said corresponding set of changes 
to determine a manner in which current image is encoded. 

71. Apparatus according to claim 70 wherein said representations each comprise a 
hierarchically-based representation of each said image. 

72. Apparatus according to claim 71 wherein said hierarchical representations each 
comprise a compositing tree of graphical objects and graphical operations. 

73. Apparatus according to claim 70 wherein said at least one member of said 
changes is other than a pixel map representation of said current image. 

74. Apparatus according to claim 73 wherein one said change comprises information 
related to a transformation matrix of changed regions in said current image. 

75. Apparatus according to claim 73 wherein one said member comprises 
information regarding content of said current image. 

76. Apparatus according to claim 75 wherein said content is selected from the group 
consisting of a plane fill and a run-length encoded data. 

77. Apparatus according to claim 76 wherein said content further comprises a pixel 
map of said current image. 
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78, Apparatus according to claim 73 wherein said one member comprises 
information regarding whether said current image is one of a background image or a 
foreground image, 

79, Apparatus according to claim 73 wherein one said member comprises 
information regarding a position and area of a region of said current image that has 
changed. 

80, Apparatus according to claims 73 wherein one said change comprises 
information regarding changes to said compositing tree. 
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